Skip to content

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374) - #5445

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate
Aug 5, 2026
Merged

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374)#5445
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5374

MemoryAnalyticsService 把每个 cube 算子映射成一个 mingo 算子的名字,调用点再把这个名字填成 matchStage[field] = {[name]: comparand}。这个形状只能表达「拿这个字段和这个值比」,别的都表达不了 —— 于是那两个需要包装比较数的算子被硬塞了进去。

notContains'$not' 编译出 {name: {$not: 'et'}}。mingo 的 $not 期望正则或算子表达式,给它一个裸标量时它不构成约束,所以谓词生成了、在管线里看得见、然后放过了全表:3 行,而 find() 是 2 行。对作者而言,一个「生成了但不起作用」的谓词和一个正确的谓词完全不可区分 —— 与 #3948 同一个放大方向,只是第三次从另一个地方到达。

这也是同一张面上第三个独立的层:#5345 裁的是没有映射的算子,#5373 裁的是比较数编码,这一条是有映射但指向了错的目标#5431 没有让它变小:那个调用点现在收到的是真实值而不是字符串化的值,但那与算子层做了什么正交。

采用的路线

issue 给了两条:调用点包装,或者让映射表返回一个结构而不是算子名。选了后者 —— issue 本身倾向它,代码也确实支撑得起。

CUBE_OPERATOR_TO_MONGO_PREDICATE 每个算子持有一个 builder,返回整个 {$op: …} 对象。于是 notContains 可以说 {$not: {$regex: …}},而「这个算子需要一个结构,可表里只放得下一个名字」这一问题消失了,而不只是这一个实例。调用点原本为 $in / $nin / $lte / $exists 长出来的那串 if,现在都是这张表里的普通行。

实测(issue 的 3 行固定数据,analytics vs find())

where 修复前 修复后 find()
{name: {$notContains: 'et'}} 3 2 2
{name: {$notContains: 'a'}} 3 0 0
{name: {$contains: 'a.p'}} 1 0 0
{name: {$contains: 'ALPHA'}} 0 1 1
{name: {$notContains: 'ALPHA'}} 3 2 2
{made_at: {$contains: '(完整 ISO)'}} 1 0 0
{code: {$in: []}} 3 0 0
{code: []} 3 0 0

同一调用点上另外三处,一并关掉了

不是扩范围 —— 要写出一个正确的 notContains,这三条每一条都必须先裁掉:

issue 里标为「未实测」的两条,已经裁了

  • 'inDateRange': '$gte':今天什么都编译不出来。没有任何 MONGO_TO_CUBE_OPERATOR 条目降级到这个名字,timeDimensions 走的是 Stage 2、根本不经过这个函数,而两个出口都只消费 normalizeFilters 的输出。所以它是死的 —— 并且万一被走到还是错的(用单边 >= 回答一个双端区间,这一点它自己的注释也承认了 "Will need special handling")。删掉了,同时删掉旁边同样是死的、并且方向反了的 'notSet': '$exists'
  • opMap[operator] || '$eq':出于同样的理由不可达 —— 但只是在有人拓宽词表之前,而 driver-memory 的 analytics 面静默丢弃大半个 filter:$or/$not 整条丢,$between/$startsWith/$null/$regex 因无 cube 映射而丢 —— 聚合结果被放大 #5345 恰恰刻意把拓宽做成了对 MONGO_TO_CUBE_OPERATOR 的一行编辑。所以不只是删掉:那张表现在是 as const,谓词表的键类型是从它派生出来的算子 union,于是那一行拓宽编辑编译不过,直到对应的谓词写出来为止。剩下的那个 throw 是全域性的兜底(totality floor),不是 fallback。

这一点是本 PR 长期价值的所在:兜底不再是「不可达」,而是「不必要」—— 全域性被证明了,而不是被防守。对 AI 写的元数据尤其重要:宽容的消费端正是 AI 生成的错误藏身并繁殖的地方。

测试

放在共享的 conformance 文件里,紧挨着 #5345 的 shape 表和 #5373 的比较数类型表,作为第三条轴,持有同一个不变量:与 find() 一致,或者拒收,绝不给出第三个更安静的答案。

外加「declared = enforced」那一半:ANALYTICS_FILTER_CAPABILITIES 声明的每一个算子都被驱动着走两条路并必须一致,且探针必须至少排除一行(否则「一致」什么也证明不了 —— 那正是本 bug 的形状)。往词表里加算子却没有可用的降级,现在会在这里失败,而不是发布一个安静的错数字。

只回退源码改动(保留测试),新断言失败 14 条:

 × $notContains excludes the rows that contain the comparand
 × a $notContains that matches every row selects none of them
 × $contains treats a metacharacter as a literal
 × $contains is case-insensitive, as the live path is
 × $notContains is case-insensitive, as the live path is
 × $contains matches a mixed-case comparand
 × $contains does not rewrite its pattern into a datetime storage form
 × $notContains does not rewrite its pattern either
 × $notContains over a column holding nulls
 × an empty $in selects nothing
 × an empty implicit-equality list selects nothing
 × $contains and $notContains partition the table for every comparand
 × every operator this face DECLARES compiles to a predicate that agrees with find()
 × the emitted $match wraps the negation around a pattern instead of a bare scalar
 Tests  14 failed | 74 passed (88)

各门实际输出:

pnpm --filter @objectstack/driver-memory typecheck   -> tsc --noEmit, 无输出
npx eslint packages/plugins/driver-memory/src        -> LINT OK
pnpm --filter @objectstack/driver-memory test        -> Test Files 17 passed (17) / Tests 512 passed (512)
pnpm --filter @objectstack/service-analytics test    -> Test Files 47 passed (47) / Tests 745 passed (745)

packages/runtime 的 suite 在本 worktree 里因为工作区依赖未构建而整文件 import 失败(Failed to resolve entry for package "@objectstack/objectql" 等)。把本改动 stash 掉后失败完全相同,与本 diff 无关。

范围之外,已单独立项

operatorToSqlgenerateSql 一个字节都没动 —— #5433 是同一类缺陷在另一个出口上,刻意不打包在一起。packages/specservice-analytics 同样未触碰。

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7


Generated by Claude Code

…dicate that excludes rows (#5374)

`MemoryAnalyticsService` mapped each cube operator to the NAME of a mingo
operator, and the call site filled that name in as
`matchStage[field] = {[name]: comparand}`. That shape can express "compare this
field to this value" and nothing else, so the two operators that need to WRAP
their comparand were pushed through it anyway.

`notContains` -> `'$not'` became `{name: {$not: 'et'}}`. mingo's `$not` takes a
regex or an operator expression; handed a bare scalar it constrains nothing, so
the predicate was emitted, appeared in the pipeline, and passed the whole table:
3 rows where `find()` returns 2. A predicate that is emitted and inert is
indistinguishable from a working one at the author's end — the same amplifying
direction as #3948, arrived at a third way. #5345 ruled on operators with NO
mapping, #5373 on the comparand ENCODING; this is a mapping pointing at the
wrong target, and #5431 did not shrink it (that call site now receives a real
value, which is orthogonal to what the operator layer does with it).

Route: let the map return a STRUCTURE rather than an operator name, which the
issue prefers and the code supports cleanly. `CUBE_OPERATOR_TO_MONGO_PREDICATE`
holds a builder per operator that returns the whole `{$op: …}` object, so
`notContains` can say `{$not: {$regex: …}}` and the CLASS of "this operator
needs a structure and the table can only hold a name" is gone rather than this
one instance. `$in`/`$nin`/`$lte`/`$exists`, which the call site had grown an
`if` chain for, are ordinary rows in that table now.

Measured on the issue's 3-row fixture, analytics vs `find()`:

| where                            | before | after | find() |
|----------------------------------|--------|-------|--------|
| {name:{$notContains:'et'}}       |      3 |     2 |      2 |
| {name:{$notContains:'a'}}        |      3 |     0 |      0 |
| {name:{$contains:'a.p'}}         |      1 |     0 |      0 |
| {name:{$contains:'ALPHA'}}       |      0 |     1 |      1 |
| {name:{$notContains:'ALPHA'}}    |      3 |     2 |      2 |
| {made_at:{$contains:'<full ISO>'}}|     1 |     0 |      0 |
| {code:{$in:[]}}                  |      3 |     0 |      0 |
| {code:[]}                        |      3 |     0 |      0 |

Three more defects at the same call site fall inside this fix and are closed
with it, because writing a correct `notContains` requires settling each:

- `contains` was the right operator with the comparand handed in RAW, so it was
  neither escaped (`.` matched any character) nor case-folded, while the live
  path escapes and matches `/…/i`. Leaving that would have made the two
  non-complementary in a new way — `alpha` would be in BOTH answers. The rule is
  now borrowed from the driver (new narrow `filterSubstringPattern`, alongside
  `filterComparandStorageForm`) rather than re-derived, per #5240.
- An operand that is NOT a comparand went through the storage-form conversion
  anyway, so on a declared `datetime` column a `$contains` PATTERN was rewritten
  into canonical form and then matched rows `find()` does not match. The builder
  input carries both lists, the same split `normalizeFieldOperators` makes
  (#4047).
- The call site's `values.length > 0` guard meant an empty `$in` emitted no
  predicate at all and widened to the whole table. A list operator taking the
  whole list has nothing to guard.

The two items the issue flagged as unmeasured, settled:

- `'inDateRange': '$gte'` compiles to NOTHING today — no `MONGO_TO_CUBE_OPERATOR`
  entry lowers to that name, `timeDimensions` never reaches this function, and
  both exits consume only `normalizeFilters` output. Dead, and wrong if it ever
  had been reached (a one-ended `>=` for a two-ended range, which its own
  comment conceded). Deleted, with the dead-and-inverted `'notSet': '$exists'`
  beside it.
- `opMap[operator] || '$eq'` is unreachable for the same reason — but only until
  someone widens the vocabulary, which #5345 deliberately made a one-line edit
  to `MONGO_TO_CUBE_OPERATOR`. So it is not merely deleted: that table is `as
  const`, the predicate table is keyed by the operator union derived from it,
  and the widening edit now FAILS TO COMPILE until the predicate exists. The
  remaining throw is a totality floor, not a fallback.

Tests go in the shared conformance file beside the #5345 shape table and the
#5373 comparand-type table, as a third axis with the same invariant: agree with
`find()`, or refuse. Plus the "declared = enforced" half — every operator
`ANALYTICS_FILTER_CAPABILITIES` declares is driven through both faces and must
agree, with a probe that must exclude at least one row, so an operator added to
the vocabulary without a working lowering fails here instead of shipping a
quietly wrong number. Reverting only the source change fails 14 of the new
assertions.

Out of scope, filed not fixed: #5440 (two operators on one field clobber each
other — the `$match` assembly layer, still broken after this), #5442
(`flattenFilterCondition` spreads an array comparand for every operator), #5444
(the `generateSql` exit emits `LIKE 'et'` with no `%` wildcards — filed as a
sub-issue of #5433, whose completion scope it falls inside). `operatorToSql` and
`generateSql` are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 11:55am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-memory.

9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-memory)
  • content/docs/deployment/vercel.mdx (via @objectstack/driver-memory)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-memory)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-memory)
  • content/docs/permissions/authentication.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/index.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-memory)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-memory)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-memory)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review August 5, 2026 12:02
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit 01c0bae Aug 5, 2026
24 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5374-notcontains-inert-predicate branch August 5, 2026 12:10
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 6, 2026
…in 滞后、死代码删除复核 (objectstack-ai#5513) (objectstack-ai#5645)

2026-08-05 跑完一整条 filter 缺陷链(objectstack-ai#5363 / objectstack-ai#5366 / objectstack-ai#5368 / objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445,
cloud#1117)后回看,六处在那一轮真实咬过人或真实救过场的规程,SKILL 里没有对应条目。
六条各落在 issue 指定的节内,**纯增补**:111 行插入、0 行删除,既有条目(objectstack-ai#5501 的接力
模式、objectstack-ai#5522 的座位模型、objectstack-ai#5630 的 assertEngineDeleteDispatch 条款)一字未动。

落点与要点:

1. **Multi-repo,rule 2 之后**「pin 滞后」——`Blocked-by:` 只保证上游已合并,姊妹仓还有
   第二个读数:本仓 pin 是否覆盖那个 commit。cloud#1116 的裁决落于 framework objectstack-ai#5368
   (`9c5abf4e9`),而 cloud 的 `.objectstack-sha` 未覆盖它,于是 `TursoDriver` 有一个
   方向反了的分叉窗口(fail-closed 一侧先到)。规程:派发前核祖先关系;未覆盖则 dev 在
   PR 正文留档窗口与方向,⛔ pin bump 不做 rider。
2. **step 3** 末「阻塞解除后重新定价」—— 前一单合入会改变后一单的成本模型,方向不止一个
   (本轮变便宜、没变、成本估计过期各有实例)。两个动作配对:派发前一单时带必答项
   「你的改动是否让 #X 变简单 / 变难 / 不必要 / 无影响」,派发被延后那单前用该回答重读
   其选项与成本估计。
3. **step 5** 派发令「多面组件的测试落点」—— 同一契约 ≥2 实现面时,新用例进共享一致性
   覆盖而非独立文件(原话照录)。附 objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445 三条正交轴共用一条不变量。
4. **step 7 清单**「收益穿过它必经的那道边界之后还在吗」—— 判据是价值主张是否依赖下游
   如实转发;实例即 objectstack-ai#5423(4xx 直通曾整条替换 ≥500 字符正文,`code` 到了正文没到)。
5. **step 7 清单**「死代码删除的复核」——「这是死代码」是断言而非能从 diff 读出的事实,
   PM 在 origin/main 独立核一次引用面再 ACCEPT(查法用 Operational notes 6:notes 6 说
   怎么查不假阴性,本条说什么时候必须查)。
6. **step 8** 升级门槛之后「带前提的裁决」—— 分歧关键是可被代码证伪的事实时,第三档 =
   裁决 + 前提验证要求 + 「前提不成立报 fork,不许硬做也不许悄悄改选」禁令,三件缺一
   不可;缺第 3 条即退化为无人裁决且无读数显示。

实施时两处核实结果与 issue 正文不同,成文按核实后的事实写:

- issue 的附带论断「没有任何闸门在量这个 pin 滞后」**不成立** —— cloud 的
  `scripts/check-pin-staleness.sh`(test.yml 以 `continue-on-error` 跑)每次 CI 都报两个
  pin 各落后 main 多少 commit,advisory 是**有意设计**(`--max-behind` 需显式传)。它答
  的是「落后多少」,不是「是否覆盖我这条裁决 commit」;成文因此指向该脚本,并只把后一个
  问题留给派发前的祖先判断。据此**未**另立「无闸门」的发现单。
- 第 4 条的 rest-server 缺陷本身已由 objectstack-ai#5423 按「截断而非替换」修掉,成文改用过去时并注明,
  以免后来的读者去找一个已不存在的活 bug;该条要补的是**复核清单的缺口**,与代码是否已修
  无关。

第 1 / 3 条按 issue「未验证的部分」的克制写入适用判据(前后单共用同一契约或数据表示;
组件对同一契约有 ≥2 实现面),形态迥异的批次(纯 UI、纯文档)明确不强加。

验证:`node scripts/check-nul-bytes.mjs --self-test` + 全仓扫描绿(48 断言 / 5537 文件);
改动文件自扫 `grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'` 零命中,并用邻近词反查证伪
「扫描器坏了」;`check:docs-audit-scope` 绿;markdown 结构核对(强调标记成对、代码围栏
16 个偶数、嵌套围栏缩进对齐)。

Claude-Session: https://claude.ai/code/session_01GX3sL71LFq8m2usg6VqTSE

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: os-zhuang <hr@objectstack.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

driver-memory analytics 面的 $notContains 编译成裸 mingo {$not: 'x'},该谓词不约束任何行 —— 结果被放大到全表

2 participants